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Abstract 

We discuss some applications of signature quantization to the representation theory of compact Lie 
groups. In particular, we prove signature analogues of the Kostant formula for weight multiplicities 
and the Steinberg formula for tensor product multiplicities. Using symmetric functions, we also find, 
for type A, analogues of the Weyl branching rule and the Gelfand-Tsetlin theorem. These analogues 
involve a g-analogue of the Kostant partition function. We show that in type A, this q-analogue is 
polynomial in the relative interior of the cells of a complex of cones. This chamber complex can be 
taken to be the same as the chamber complex of the usual Kostant partition function. We present the 
case of A2 as a detailed example. 

1 Introduction 

The results described in this note are closely related to an article of Guillemin, Sternberg and Weitsman 
[ 7 ] on signature quantization. 

Denoting by V\ the irreducible representations of a complex semisimple Lie group, the work of Guillemin, 
Sternberg and Weitsman on quantization with respect to the signature Dirac operator involves the "twisted" 
representations V\ = V\s <X> Vs f° r strictly dominant weights A. They give a formula for the multiplicities 
of weights in those representations which is very similar to the Kostant multiplicity formula, but involves 
the q = 2 specialization of a g-analogue K q of the Kostant partition function, rather than the usual (q = 1) 
partition function. This g-analogue arises from the work of Agapito [ 1 1 and Guillemin, Sternberg and 
Weitsman [ 7 ] in the study of the twisted signature of coadjoint orbits 

We explore further the structure of these representation. We explain how they decompose into irreducible 
representations and show that it is possible to decompose a tensor product of twisted representations into 
twisted representations again. There is a formula very analogous to that of Steinberg for the multiplicities of 
the factors in the product, which again involves if 2- An interesting feature of this formula and the analogue 
of the Kostant multiplicity formula of is that they are free of the 5 factors of the usual formulas for the 
irreducible representations. For type A, we can write down the characters of the V\ in terms of Schur 
functions, and we find a branching rule for restricting the representation V\ of GL^C to GLfc_iC. By 
iterating this rule, we develop a Gelfand-Tsetlin theory for the twisted representations of GL^C. 

Finally, we describe the structure of the (/-analogue K q of the Kostant partition function. We show that 
for the root system A n , this g-analogue is polynomial in the relative interior of the cells of a complex of 
cones, of degree Q) with coefficients in Q[q] of degree ( n J ). This chamber complex can be taken to be 
the same as the chamber complex of the usual Kostant partition function. We present the case of A2 as a 
detailed example. 
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1.1 Quantization with respect to the signature Dirac operator 



1.1 Quantization with respect to the signature Dirac operator 

A symplectic manifold (M, u) is pre-quantizable if the cohomology class of uj is an integral class, i.e. is in 
the image of the map H 2 (M, Z) — ► H 2 (M, R). This assumption implies the existence of a pre-quantum 
structure on M: a line bundle, L, and a connection, V, such that curv(V) = uj. If g is a Riemannian 
metric compatible with uj, then, from g and uj, one gets an elliptic operation C : S + — > S~, the spin-C 
Dirac operator, and, by twisting this operator with L, an operator 0^ : S + ®L -> S~ (g> L. If M is 
compact one can "quantize" it by associating with it the virtual vector space 

Q{M)= Index (1) 

Moreover if G is a compact Lie group and r a Hamiltonian action of G on M one gets from r a represen- 
tation of G on Q(M) which is well-defined up to isomorphism (independent of the choice of g). 

The results described in this note are closely related to two theorems in the article In this article the 
authors study the signature analogue of spin-C quantization: i.e. they define the virtual vector space Q 
by replacing $ c by the signature operator sig , and prove signature versions of a number of standard 
theorems about quantized symplectic manifolds. The two theorems we'll be concerned with in this paper 
are the following. 

1. Let G = (S" 1 ) 71 and let M be a 2n-dimensional toric variety with moment polytope A C M. n . 
Then, for spin-C quantization, the weights of the representation of G on Q(M) are the lattice points, 
€ A n Z n , and each weight occurs with multiplicity 1. For signature quantization the weights are 
the same; however, the weight (3 occurs with multiplicity 2™ if lies in Int(A), with multiplicity 
2«,-i ^ j t jj es Qn a f acet> an( j ? j n g enera i ; with multiplicity 2 n ~* if it lies on i facets. Further details 
can be found in the work of Agapito 

2. Let G be a compact simply connected Lie group, A a dominant weight and 0\ = M the coadjoint 
orbit of G through A. In the spin-C theory, the representation of G on Q(M) is the unique irreducible 
representation V\ of G with highest weight A; however, in the signature theory, it is the representation 

V x = V x . p V p , (2) 
where p is half the sum of the positive roots. (This is modulo the proviso that A — p be dominant.) 

The article Q also contains a signature version of the Kostant multiplicity formula. We recall that the 
Kostant multiplicity formula computes the multiplicity with which a weight, \x, of T occurs in V\ by the 
formula 

J2(-l)^K(a(X + p)-(f, + p)) (3) 

where W is the Weyl group, \a\ is the length of a in W, and K, the Kostant partition function (described 
below in Definition ll.lt . The signature version of the Kostant multiplicity formula computes the multi- 
plicity fh\(p) with which the weight p appears in V\ by a similar formula: 

mx{p)= ^(-l)HK 2 KA)-/x) (4) 

where K2 is the q = 2 specialization of a new g-analogue of the Kostant partition function, described 
below. 



1.2 The Kostant partition function and its q- analogues 
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Our initial goal in writing this paper was to give a purely algebraic derivation of this result; however 
we noticed that there are V\ analogues of a number of other basic formulas in the representation theory 
of compact semisimple Lie groups, in particular, an analogue of the Steinberg formula and, for GL^C, 
analogues of the Weyl branching rule and the Gelfand-Tsetlin theorem. 

1.2 The Kostant partition function and its g-analogues 

We start by introducing the Kostant partition function. 

Definition 1.1 The Kostant partition function for a root system <3?, given a choice of positive roots <5 + , is 
the function 

' " (5) 



K(n) = \{(k a ) ae * + ef#+l : Y. k « a = ^} 



i.e. K(n) is the number of ways that p, can be written as a sum of positive roots (see [5]). 

Note that K(p) can also be computed as the number of integer points inside the polytope 

Q^ffcU^R?; 1 : ^/c Q a = /i}. (6) 

We can write down a generating function for the K(p) that is very similar to Euler's generating function 
for the number of partitions (see [5 Section 25.2]): 

E^=llr^' ( 7 > 

The classical ^-analogue K q (p) of K(p), due to Lusztig |9], keeps track of how many times the roots 
appear: 

K q (n)= E 3 E *"' (8) 
corresponding to the generating function 

n (e^) = n w 

The g-analogue K q (p) that interests us here is the one that counts the integer points of according to 
how many of the k a 's are nonzero: 

K q {n)= E ?' {fcQ>0}1 - ( 10 ) 
In terms of generating functions, this translates to 

j>,(„)e"= n (i+izA = n 1 + 1 to / e ! )e ° ■ CD 

A* oG<3?+ \ m>l I ag«I> + 
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2 An analogue of the Kostant multiplicity formula 

We are working in the context of a complex semisimple Lie algebra g with root system <I>, choice of positive 
roots $ + , and Weyl group W ; p is half the sum of the positive roots (or the sum of the fundamental 
weights). For a dominant weight A, we denote by V\ the irreducible representation of g with highest 
weight A. We will call a weight A strictly dominant if A — p is dominant. We will use the notation A + 
for the set of dominant weights, and Ag for the set of strictly dominant weights. For a strictly dominant 
weight, we define the representation 

V x = V x . p V p (12) 

and its character 

X\ = Xv x _ p ®v p = Xa-p ' Xp ■ (13) 

The following theorem of Guillemin, Sternberg, and Weitsman [7 1 provides a formula for the multiplicities 
of the weights in the weight space decomposition of V\. This formula is very similar to the Kostant 
multiplicity formula ©, but uses the q = 2 specialization of the (/-analogue of the Kostant partition 
function K q {p) introduced above, instead of the usual Kostant partition function. The formula for the V\ 
multiplicities further distinguishes itself from the Kostant formula by being free of the p factors. 



Theorem 2.1 (Guillemin-Sternberg- Weitsman |7|) Let X be a strictly dominant weight. Then the mul- 
tiplicity of the weight v in the tensor product V\ = V\- p ® V p is given by 

m x (u) = dim [V x ) v = (-1) M ^MA) - v) , (14) 
where \u\ is the length of lo in the Weyl group. 



Proof. We give a simple proof here using the Weyl character formula. This formula expresses the charac- 
ter X\ °f V\ as the quotient 

Xa = ^ , (15) 



where = ^ (_i)l a, l e a, W . For p, we get the nice expression |5] Lemma 24.3] 

a p = n ( e ° /2 - e ~ a/2 ) = eP n i 1 - e ~ a ) . ^ i6 ) 



which means, in particular, that we get 



e 2p Yl (1 - e- 2a ) 



\„ - X ^ ™ — - eP II ( 1 + e_Q ) • ( 17 ) 



2 An analogue of the Kostant multiplicity formula 



5 



Thus, for A strictly dominant, 



l + e- a 



£(_1)M^(A) JJ i 



(18) 



Extracting the coefficient of e v on both sides gives fT3V 



(19) 



The next step will be to use a formula due to Atiyah and Bott for the characters of the V\ and V\ to break 
down V\ into its irreducible components and find their multiplicities. The Atiyah-Bott formula 013) gives 



the character of Vp as 



* = e ^ n 



j£W a£$ 



o;(o) 



(20) 



Remark 2.2 We can deduce this formula from the Weyl character formula (equation (I15t ) by first observ- 
ing that 



no 



n a -""■) n (!-<") 



q e A + 

w(a) £ A+ 



a £ A + 
ii)(a) £ A- 



n n ((^ a -i)- a ) 



a £ A + 
w(a) £ A H 



a £ A+ 
U)(a) £ A_ 



(— l)'^ aeA+ : UJ ( a ) eA -}\ H e a H (l-e-°) [] (l-e" a ) 



a £ A+ a £ A + 
u(a) £ A_ w(a) £ A + 



a £ A+ 
w(a) £ A_ 



4)M e £{aeA + : w(a)eA_} JJ (\ _ 



(21) 



«£A4 



57«ce ?/2e number of positive roots that are sent to negative roots under u is the same as the length \u\ of lo 
in the Weyl group. 



On the other hand, 
6-u(5) -- 



a£A H 



«£Aj 



E« + 5 E 



a £ A + 
w(a) £ A H 



o 



a £ A + 
a>(a) £ A- 



1 y, 

a £ A + 
\ u(a) £ A 



a: 



\ E 



Q £ A + 

£ A-/ 



= ^{a G A+ : w(a) G A_} 



(22) 
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2 An analogue of the Kostant multiplicity formula 



Combining (1211) with (1221) gives 

Yl (l-e~ w(a) ) = (-i)Me' 5 -^' 5 ) [J (l-e _a ), (23) 
oeA + ' qgA+ 

ancf we can translate Weyl's character formula into the Atiyah-Bott formula using this equation. 
For any lo € W, 

X P = e' n ( 1 + e " Q ) 

W(P) II (l + e~ u(a,) ) , (24) 



e 



since characters are invariant under the Weyl group action. Using this and the Atiyah-Bott formula, we can 



write 1 



wGW aG$+ 



V TT I V 

L / J. J. ]_ _ g— w(a) Z— >■ 

where as before, a/ = £ a . This gives 



e -«(ar) 



wgW »G<E>+ /C$_, 



Q6J 



*/) TT - I 

11 1 _ g-w(a) I 
aG$+ / 



*a - E E ^ (A -°" n T- 1 *! ■ < 26 » 



Letting, \j = A — aj, we observe that if A/ is dominant, the Atiyah-Bott formula tells us that 

e n t^t- 



-uiipi) 



(27) 



is the character of the irreducible representation V\j , so that 

Xx = £ X A/ and F A = Va_ p ® F„ = V Aj (28) 



if all the A/ are dominant. 



1 Alternatively, we can obtain equation 1251 from equation 1 1 Si by observing that for to £ W, 
( TT ±±£l \ TT 1 + e ~" ( " ) / W«l TT 1 + e " 



3 A TENSOR PRODUCT FORMULA FOR THE V\ 
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Finally, since aj and aj> can be equal for different subsets / and certain highest weights appear multiple 
times in the above sums. For the weight \i = Xj = A — aj, we will get V p as many times as we can write 
aj = A — fx as a sum of positive roots, where each positive root appears at most once. Hence 

V\ = ^2 P(X — fj,)V/j,, (29) 
where the sum is over all p such that (j, = Aj for some I, and P{y) is given by 

v aG<3>+ 

Remark 2.3 David Vogan pointed out to us that this decomposition is well-known and can be deduced 
from the Steinberg formula. For type A n , the number of distinct fi 's in the above sum is the number of 
forests of labelled unrooted tree on n + 1 vertices lS\ \13V . 



3 A tensor product formula for the V\ 



We will derive here an analogue of the Steinberg formula for the V\. Given two representations V\ and 
V/j,, the problem is to determine whether their tensor product V\ <8> Vp, can be decomposed in terms of V^'s. 
This is readily seen to be the case, as 

V\®% = (Vx- p ®V p )®(V ll - p ®Vp) = (Vx-p®V p ®V li -.p)®V p . (31) 

Breaking up V\- p <8> V p <8> Vfi-p into irreducibles V~ and tensoring each factor with V p yields factors 
V-y (g> V p = Vy+ p . Thus for strictly dominant weights A and p,, we can write 

V x ® % = E < 32) 

for some nonnegative integers . 

Theorem 3.1 For A, fx and v strictly dominant weights, the tensor product multiplicity NV of V v in 
V\ ® Vp is given by 



Proof. Starting from the equation V\ ® Vp = ./Vj^VL, we can use equation (fT8t to write 

l + e" Q „ r™ w,^ tt l + e- Q 



E( _ 1) M e .(A) n ^ • x,= e ^E(-D H ^ n f 

Cancelling terms and using Theorem l2.1l to write down the character Xp yields 

E (-i) M ^ (A) • E E (-i) h *»(*(aO - /?) = E < E t- 1 ) 1 ^ 
EE E(- 1 ) H+H ^(^)-/ 3 ) eW(A)+/3 = EEH) H ^ eT 

/3 ueWusW z^eAlj T 6 w 



r(iz) 
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4 Links with symmetric functions in type A 



Substituting 7 = uj(X) + [5 on the left hand side, and 7 = t{v) on the right hand side gives 



EE Et-^^w+^A)-^^ E Et- 1 ) 1 ^ 

7 wGW ctGW 7 conjugate 

to a strictly 
dominant weight 



I /vr 1(7) e 7 



and extracting the coefficient of e 7 on both sides yields 



£ ^ (-l)l'H K 2 {a(p) + w(A) - 7) = E ( -1 ) |T| ■ ( 34 ) 

Now, since iV^T ^ vanishes unless t -1 ^) is strictly dominant, all the terms in the sum on the right hand 
side vanish except for the one where r is the identity (i.e. the term where 7 = v), and we get the result. ■ 

If we denote by N£ the multiplicities of the irreducible representations V v in the tensor product V\ ® V^, 
defined by 

Vx®V IM =Y, N *> V >" (35) 
ueA+ 

then we can write down the tensor product multiplicities NY for the decomposition of (g> Vp, into V v 's 



in terms of the N% as follows: 



v/3GA+ 



E n I,p n Ip-p v A ^ 

v fleA+ 7GA+ 

E E n Lp,p n Ip-p^ p 

/3eA+ 7GA+ 

E E<p,p^7-^- 



p 



VGA+ /3GA+ 



so that for strictly dominant v, 



N xp= E n Lp,p n p7-p- w 

/3eA+ 



Remark 3.2 In type A, there is a combinatorial interpretation for the coefficients N% in terms of shifted 
Young tableaux: they are given by a shifted analogue of the Littlewood-Richardson rule (see M5V ). 



4 Links with symmetric functions in type A 

As for the weight multiplicities and Clebsch-Gordan coefficients, there is a link between the character 
products Xa = Xx-S ' Xs and symmetric functions in type A, again in terms of Schur functions. 



4 Links with symmetric functions in type A 
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The character of the irreducible polynomial representation V\ of GL^C, where we now think of A as 
a partition with k parts (allowing the empty part) is the Schur function s\{x\, . . . ,x k ). We will call a 
partition strict if all its parts are distinct (corresponding to a strictly dominant weight). Thus we have that, 
for GL fc C, 

X\ = Xxs-Xs = ax-s(xi,...,Xk)3s(xi,...,x k ) : (37) 
for any strict partition A. The weight 5 corresponds to the partition (k — 1, k — 2, ... ,1,0). 

Remark 4.1 We can also write the characters ofV\ in terms of Hall-Littlewood polynomials. Following 
M0\ III.l and 111.2], for partitions of length at most k with empty parts allowed, let 

1 1-f 



i=l 

and define 



v\(t) = ]Jv mi {t) (39) 



where mi is the number of parts of X equal to i. 

The Hall-Littlewood polynomials are the symmetric polynomials defined by 



P\(xi, ...,x k ;t) = —r-rRxix!, . . .,x k ; t) , (40) 



where R\ is the symmetric polynomial 



R x (x 1 ,...,x k ;t)= Y, w Wx ■ ■ ■ 4" II Xi x . Tj. | • ( 41 > 
ue& k \ i<j X% Xj 

We can rewrite Rx as 

R X ( X1 , ...,x k ;t)=J2u J . U -4 k II a" til 1 • < 42 > 

For a strict partition A with k parts, v\(—l) = 1 and then, 

P\(exp(ei), . . . , exp(e k ); -1) = V" u ■ ( exp(Aiei H h \ k e k ) TT ^ + ex P( e j — e ±) 

w£6i \ aeA + 

= V TT + 

Z—/ 11 ]_ _ e — w(a) 

= X A -5 • X 5 (43) 
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5 A BRANCHING RULE FOR THE V\ IN TYPE A 



from the Atiyah-Bott formula (equation (I25t ). So the character product X\s ' Xs can be thought of as the 
t = — 1 specialization of the Hall-Littlewood polynomial P\. 

The results of the following sections can be deduced from this link with Hall-Littlewood polynomials, but 
we will rather use the Schur function expression (137 1 for the characters. This makes the proofs a bit more 
technical but avoids the heavier machinery of Hall-Littlewood polynomials. 

5 A branching rule for the V\ in type A 

We have seen that the representations V\ behave somewhat like irreducible representations, in that tensor 
products of them can be broken down into direct sums of V v 's again, and that the multiplicities in those 
decompositions as well as in the weight space decomposition are given by formulas very similar to those 
of Kostant and Steinberg in the irreducible case. The Weyl branching rule (see [ 5 1 for example) describes 
how to restrict a representation V\ from GL^C to GLfc_i<C. This rule can be applied iteratively and 
prodides a way to index one-dimensional subspaces of V\ by diagrams (Gelfand-Tsetlin diagrams [6]) that 
is compatible with the weight space decomposition. It is natural to ask whether the representations V\ of 
GLfcC are also well-behaved under restriction, or in another words, if there is an analogue of the Weyl 
branching rule for the V\ in type A. 

For two partitions p = (pi, . . . , p m ) and 7 = (71, ... , 7 m _i), we say that 7 interlaces p, and write 7 < p,, 
if 

Ml > 71 > M2 > 72 > > • • • > Mm-1 > 7m- 1 > Mm • 
For two such partitions p and 7 such that 7 <\ p, we define 

V(m,7)= |{«e{l,2,...,m-l} : Pi > 7* > Mi+i} | • (44) 
In other words, V(/x, 7) is the number of 7$ that are wedged strictly between pi and Mi+i- 

Theorem 5.1 The decomposition of the restriction of the representation V\ of GL^C to GL^,_iC into 
irreducible representations of GLfc_iC is given by 

R«gl£c^= © 2 v (V)vt. (45) 

Proof. We will argue using characters and the fact that those can be written in terms of Schur functions. 

We saw above (equation d37t ) that the character of the representation V\ of GL^C is the product of Schur 
functions s\s(xi, . . . , Xk) ss{x\, . . . , x^). We obtain the character of the restriction of V\ to GLfc_iC 
by setting the last variable x^ equal to 1. Now if we have a Schur function in two sets of variables 
x = (x\,X2, ■ ■ •) and y = (yi, y2, ■ ■ .) with the ordering x\ < X2 < ■ ■ ■ < y\ < 2/2 < • • • > then we have 
the identity 

s\(x, y) = ^2 s ^ • (46) 

This is simply saying that we get a semistandard Young tableau of shape A with entries in x and y by first 
filling a subtableau p with entries in x and then the remaining skew-shape with entries from y. In our case, 
with x = (x\, . . . , Xk-i) and y = 1, this yields 

s A (xi,... = ^2s fl (x 1 ,...,x k ^ l )s x / IJ ,(l). (47) 
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However, we have sw^l) = unless A//i is a horizontal strip, in which case it is equal to 1, and also 
s fl (xi, . . . , Xk-i) = if \i has k parts or more. Hence 

s x (xi, . . . ,x k -i, 1) = y^s^xi, . . . ,x k -i) (48) 

where the sum is over all \i with at most k parts that can be obtained from A by removing a horizontal strip. 
The set of such /i's is seen to be the set of partitions that interlace A, so 

sx(x 1 ,...,x k -i,l) = ^2 s^{x 1 ,...,x k -i). (49) 

/J< A 

Also, in the Xi = exp(ei) coordinates, 

x 5 = e s n 

aSA + 

= x\- x x\- 2 ---x^ 11 (l + - 

l<i<j<k ^ ' 

= n ( x i+ x j)- ( 5 °) 

l<i<j<k 

This can also be deduced from the classical definition of the Schur functions in terms of determinants 
fT4l Section 7.15], since s$(xi, . . . ,Xk) is the ratio between the Vandermonde determinant in variables 
x\, . . . , x\ and the Vandermonde determinant in x\, . . . , Xk- Thus, 

fc-i 

8\-s(xi,... ,X k -i ,1)S 5 (X 1 , ... ,Xjfc_i, 1) = ^ S m( X 1' ■ ■ ■ jZife-l) II (Xi + Xj) ^{(Xi + l) . (51) 

fi<i\-8 l<«<j<fc— 1 *=1 

We recognize the product ni<i<i<A.-i( x * + x i) as tne ^chur function ss(x\, . . . ,Xk-\) (where 5 now 
corresponds to the partition {k — 2, k — 3, . . . , 1, 0) with k — 1 parts), and the product nfct" C 35 * + -Q as 
the sum (eo + e\ + ■ ■ ■ + e k -\) of elementary symmetric functions in the variables x±, . . . , x k -i- A dual 
version of the Pieri rule [ 14 Section 7. 15] describes how to break down the product of a Schur function 
with an elementary symmetric function into Schur functions: 

S[i &m — ^ Su ; (52) 

where the sum is over all v obtained from \i by adding a vertical strip of size to, i.e. over the v such that 
[i C v and the skew-shape v I \i consists of to boxes, no two of which are in the same row. As we are 
working in k — 1 variables, the s u with more than k — 1 parts vanish, so we can add the further constraint 
that the vertical strip be confined to the first k — 1 rows (we will say such a vertical strip has height at most 
k — 1). This gives 

S\-s(xi,...,X k -l,l)ss(Xl,...,X k -l,l) = ^ ^ S u (xi, Xk-l) S S (XI, 3C fc _i) 

H<i\— S v 

X A (a?i,...,Xfc_i,l) = ^Xu+si^i, - ■ ■ ,x k -i) (53) 

\x < A— 8 v 
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where the sum is over all the u that can be obtained from /i by adding a vertical strip of size and height at 
most k — 1. We can rewrite this as 

X x (xi,...,x k -i,l) = ^2x u (xi,---,Xk-i) (54) 

where the sum is over all strict partitions v such that v — 8 can be obtained from fi by adding a vertical 
strip of size and height at most k — 1. Since the s v ss are linearly independent, we can lift this to the level 
of representations to get 

Resgf iC y A = © ©K, (55) 
with the sum over the same set of v as before. 

In order to compute the multiplicity of a given V u in Res GL * c V\, we define, for strict partitions A and 
v, n(A, v) to be the number of ways that u — 5 can be obtained by adding a vertical strip of size and height 
at most k — 1 to some partition fi such that < A — 5, so that 

Vx = © ™(A, f) it . (56) 

Note that 5 has two different meanings here: for the group GL^C, it corresponds to the partition {k— 1, k— 
2, . . . , 1, 0), while for GL^^C, it corresponds to the partition (k — 2, k — 3, . . . , 1, 0). To avoid confusion, 
we will denote the latter by 5'. 

The condition <\ X — 5 means that 

Ai - (k - 1) > yui > A 2 - (k - 2) > n% > ■ ■ ■ > X k -i - 1 > Hk-i > Afc • 

Replacing m by m + ^ = m + (k — 1 - i) = gives 

Ai - 1 > m + (k-2) > A 2 
A 2 - 1 > M2 + (k - 1) > A 3 

Afe-i - 1 > jUfc_i + (0) > A fc . 

These equations mean that the i-th part of // = yu + 5' is at least as large as the {i + l)-th part of A and 
smaller than the i-th part of A. In other words, the skew-shape X/fi' is a horizontal strip with a least a box 
in each row, or equivalently, fjf < X with the further constraints ^ < X{ for all 1 < i < k — 1. Adding a 
vertical strip to fi to get u — S is the same as adding a vertical strip to // to get i/, provided that we only 
allow adding vertical strips to fi' that result in a strict partition. It is then clear that by adding such a vertical 
strip to //, we get a strict partition v such that X/v is a horizontal strip. Conversely, it is also clear that for 
any strict u such that X/v is a horizontal strip, there is a // such that v can be obtained from // by adding 
a vertical strip. So the only summands V v for which n(A, ^) 7^ in the decomposition d56b are those for 
which v < A. 

Given such ay, we will compute n(A, y) by constructing row by row the strict partitions // = \i + <5' from 
which we can obtain za Given z^, there are three cases to consider for the possible ^[ : 

• vi = X{. In this case, since we must have < Aj, it has to be that = Aj — 1 and that we have a 
box in row i of the vertical strip. So there is only one choice for /i- . 
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• v, L = Aj + i. Then we must have fii = Aj+i < ^ < and therefore ^ = V{, so we don't have a box 
in row i of the vertical strip. Again, there is only one choice for ^ in this case. 

• Xi > Vi > Aj+i. Then we can either have /i- = i/j — 1 and have a box from the vertical strip in row 
i, or have ^ = v-i and have no box from the vertical strip in row i. So there are two possibilities for 
\J i in this case. 

We have to show that any choice of ^ that we make gives rise to a strict partition (by construction, it is 
clear that //' < A). If for some i we had ^ = then because Aj + i is at least fj,' i+1 + 1, this would mean 
that Xi is at least ^ i + 2, since Aj > Aj+i. But then A///' contains two boxes in the same column: the box 
after box pb i in row i, and the box after box \J i = in row i + which contradicts the fact that // < A 
(or equivalently, that A///' is a horizontal strip). Hence we get two choices for each instance of a pattern 
of the form Aj > > Aj+i. We called the number of such instances above V(A, v). Since the choices at 
each row are independent, we have 

f 2 V ( A >^ if v < A and v G At , 
n(A,z.) = <^ S (57) 

[ otherwise. 

from which the proposed expression for the branching rule follows. ■ 



6 A Gelfand-Tsetlin theory for the V\ in type A 

After restricting to GL^^C, we can further restrict to GLfc_ 2 C. From now on, we will assume that all 
partitions are strict. We can write 

^%lf_ 2 C^ = Re S ^ : ^(Resg^ C iC y A ) 

\z/<A / 

z/<A 

v < A V/i < f / 

= 2 V(A,,)+V(^) ^ _ (5g) 

/i < v < A 

Denoting by A^" 1 ) = A^" 1 " 1 > ■ ■ ■ > A^ > the strict partitions indexing the representations V of GL m C, 
we can iterate the branching rule until we get to GLiC : 

ResgftE Vx = 2 v ( A(fc) ' A(fc - 1) ) +v ( A( ^ 1) ' A( ^ 2) ) + - +v ( A(2) ' A(1) ) V xm . (59) 

A(i) <...<|A( fc )=A 

We will call a sequence of strict partitions of the form X^ < ■ • • < A( fe ) = A a twisted Gelfand-Tsetlin 
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diagram for A, which can be viewed schematically as 



(fc) 



A. 



(*) 



A k-i 



A 



(*) 



A 



(fe-i) 



.(fc-i) 



(fc-i) 



(2) 



,(2) 



Ai 



(i) 



with X) = Xj and each X) is a nonnegative integer satisfying 



and 



for all 1 < j < i, 1 < i < k - 1. 



A (i + D > A w > A (m) 



(60) 



(61) 
(62) 



Let be the subspace of V\ corresponding to a twisted Gelfand-Tsetlin diagram V. This subspace has 
dimension 2 V ( X> ), where 



V(P) = V(A (fc) , A (fc - 1} ) + V(A( fc - 1 \ A (fe - 2) ) + • • • + V(A (2) , A«) 



(63) 



We can also think of V(X>) as the number of triangles 



A 



(0 



A 



(0 



'3 "3+1 

A «, +1 , 

with strict inequalities A^ +1 ^ > A^ > A^ 1 ^ in the diagram V. 

We show here that Vr> lies completely within the same weight space of the weight space decomposition of 
V x . 

Consider GL&C with its subgroup Tj, of invertible diagonal matrices, and also its Lie algebra gl k C and 
its "Cartan" subalgebra tk of diagonal matrices. We have the natural basis in which weights are usually 
written 



Ji 



( 



\ 







o / 



for tfc, and also the basis 
/ 1 

h = 

V 



o 



o 



h 



o / 



\ 

o / 

\ 

o / 



Jk 







r i 



\ 

1 / 
\ 

1 / 
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which is more convenient and which we will use to do the computation. We will simply have to remember 
that Ji = I{ — to get the weights in their usual form at the end. 

We will think of the groups GL&C as included into one another by identifying GL m C with 



GL m C 











J 



Consider the element / G Ql m C and a representation of GL m C. Then we have the representation 
GLfcC — > gl(V^ <8> V p ). For v G Vjx-p an d w G V p , we have 



I ■ (v <gi w) = (I ■ v) <8) w + v <8> (I ■ w) 

y^(jU - p)j \ v <g) u> + w 



Z u >' l' ] < ' ft) 1 

since V^_ p has highest weight \i — p and V p has highest weight p. So / G s[ m C gets represented as 
In general, for 

RpoGL fc C tT__ /T\ 9 V(A( fc ),A( fc - 1 ))+V(A( fc - 1 ),A( fc - 2 ))+-+V(A( m + 1 ),A( m ))T7^ 

GL m C Vx ~ \P z v \(™) ' 



A(") <-<A(*0=A 



we will find that I G Ql m C gets represented as {YliLi ^f"^) I i n ^a( 



m) • 



Therefore, in the basis Ji, ... , Ij,, the subspace corresponding to a twisted Gelfand-Tsetlin diagram V 
has weight 

(X>« £A.p,...,X>f) 



or 



fc-i 



i=i 



i=l 



i=l 



( E • E \ m - E ■ - ■ E \ w - E *!*-") 

i=i i=i 
in the usual basis Ji, . . . , J*.. 

In other words, Vx> C (V\) ^ if 



m— 1 



An = E A i m) -E^ m " 1) . 



i=l 



(64) 



(65) 
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or, equivalently, 

m 
i=l 

Hence twisted Gelfand-Tsetlin diagrams for A correspond to the same weight if all their row sums are the 
same. So we have proved the following analogue of the Gelfand-Tsetlin theorem [6|. 

Theorem 6.1 Let A = (Ai, . . . , X k ) be a strictly dominant weight. The dimension of the representation V\ 
o/GLfeC is given by 

dimVA = ^2 v ^ (67) 
v 

where the sum is over all twisted Gelfand-Tsetlin diagrams with top row A. 
Furthermore, the multiplicity rh\{(3) of the weight (3 in V\ is given by 

ih x (0) = dim (Vx) p = 2V{V) ( 6§ ) 

v 

where the sum is over all twisted Gelfand-Tsetlin diagrams with top row A and row sums satisfying equa- 
tion (ED (or (l66t). 

Remark 6.2 We can also prove that Vt> lies completely within a weight space ofV\ using characters. The 
Schur function identity (I49t 

s\{x,y) = ^ s^x) sx/niy) (69) 
n<z\ 

in the two sets of variables x and y gives 

s x {xx,...,x k -i,x k ) = ^s M (a;i,...,a; fe _i)4 



^s At (a;i,...,a; fc _i)a;j fc A| ImI (70) 



JA]-|mI 
n 

where the sum is over all fi such that \/p is a horizontal strip (or equivalently, p such that fx <d X). We 
also have 

k-l 

Yl fa + x k ) = 4 _1 eo + x k k ~ 2 e 2 + ■■■ + x k e k _ 2 + &k-i , (71) 
i=i 

where as before, the e m are the elementary symmetric functions in the variables x\, . . . , x k . This gives 

k-l 

s\(xi,...,x k ) = ^ s^(x 1 ,...,x k ^i)x^ S ^ M s 5 (x 1 ,...,x k ^ 1 )Yx k r 1 ~ m e m 

fj,<\—5 m=0 

2_^s v (x\, . . . ,x k _i) s s (xx, . . . ,x k -i) x k ln , 

/i<lA-<5 v 

where the sum is over all v that can be obtained from p by adding a vertical strip of size and height at most 
k — 1. Now \ f/p\ = \v\ — \n\ and \5\ — \5'\ = k — 1, so we get 

X x {x 1 ,...,x k ) = ^ Yx v+Sl (xi, . . . ,rE fc _i)4 AHv+<5 ' ( 72 ) 
= E 2^)^4 AhH - (73) 

i/GA"^ : v < A 
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Iterating as before yields 

~ . V" oV(a« a^+.-.+vca^aW)- jaW| ja( 2 )|-|a(D| jaWhaC*- 1 )! m , 

Af 1 ) <.-.< A( fc )= A 
Therefore Vx> lies in the weight space with weight 

(|A«UA( 2 )MA (1) |,...JAW|-|A (fe - 1) |) , (75) 
which is the same as what we found in equations (I64t and \65\ . 

7 A chamber complex for the (/-analogue 

We will assume in this section that we are working in type A n . We will let <£ + = {a±, . . . , a^}, with 
N = ("'t )■ We will denote by C n the chamber complex for the Kostant partition function. For positive 
integer k, we will use the notation [k] for the set {1,2,..., k}. 

Definition 7.1 Let M be a d x n matrix over the integers, such that kerM n M> = 0. The vector partition 
function (or simply partition function) associated to M is the function 

4> M ■ % d — ► N 

b ^ \{x € W l : Mx = b}\ 

The condition kerM n M> = forces the set {x E N n : Mx = b} to have finite size, or equivalently, 
the set {x £ E> : Mx = b} to be compact, in which case it is a polytope Pf,, and the partition function 
is the number of integral points (lattice points) inside it. 

Also, if we let M\, . . . , M n denote the columns of M (as column-vectors), and x — (^lj ■ • • > %n) ^ ^>0' 
then Mx = x\M\ + X2M2 + • • • + x n M n and for this to be equal to b, b has to lie in the cone pos(M) 
spanned by the vectors Mj. So 4>m vanishes outside of pos(M). 

It is well-known that partition functions are piecewise quasipolynomial, and that the domains of quasipoly- 
nomiality form a complex of convex polyhedral cones, called the chamber complex. Sturmfels gives a very 
clear explanation in [16] of this phenomenon. The explicit description of the chamber complex is due 
to Alekseevskaya, Gel'fand and Zelevinskii 0. There is a special class of matrices for which partition 
functions take a much simpler form. Call an integer dxn matrix M of full rank d unimodular if every non- 
singular d X d submatrix has determinant ±1. For unimodular matrices, the chamber complex determines 
domains of polynomiality instead of quasipolynomiality [ 16 1. 

It is useful for what follows to describe how to obtain the chamber complex of a partition function. Let 
M be a d x n integer matrix of full rank d and <pM its associated partition function. For any subset 
a C {1, . . . , n}, denote by M a the submatrix of M with column set a, and let r CT = pos(M CT ), the cone 
spanned by the columns of M a . Define the set B of bases of M to be 

B = {aQ{l,...,n} : \a\ = d and rank(M CT ) = d} . 

B indexes the invertible d x d submatrices of M. The chamber complex of 4>m is the common refinement of 
all the cones r a , as a ranges over B (see [2]). A theorem of Sturmfels [ 16 1 describes exactly how partition 
functions are quasipolynomial over the chambers of that complex. 
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If we let Ma„ be the matrix whose columns are the positive roots <3?^ A "' ) of A n , written in the basis of 
simple roots, then we can write Kostant's partition function in the matrix form defined above as 

K A n (v) = (t>M An (v) . 

The following lemma is a well-known fact about Ma„ and can be deduced from general results on matrices 
with columns of O's and l's where the l's come in a consecutive block (see [ 12 1). 

Lemma 7.2 The matrix Ma„ is unimodular for all n. 

Ma„ unimodular means that the Kostant partition functions for A n is polynomial instead of quasipoly- 
nomial on the cells of the chamber complex. In general, for M unimodular, the polynomial pieces have 
degree at most the number of columns of the matrix minus its rank (see ED)- m our case, Ma„ has rank 
n and as many columns as A n has positive roots, ("J 1 ) - Hence the Kostant partition function for A n is 
piecewise polynomial of degree at most — n = Q) . 

We can now state the main result of this section. 



Theorem 7.3 The q-analogue K q (//) is given by polynomials of degree (2) with coefficients in 
degree ("J ) over the relative interior of the cells ofC n . 



of 



Proof. We start at the level of generating functions by observing that 



N 



l + (q- l)e Qi 



N 



N 



u— 

i=l 



\i=i 



\i=l 



IC[N] 



IC[N] V 



where aj = a.j 

iei 



Extracting the coefficient of e M in the previous equation gives 

IC[N] 



(76) 



(77) 



Now, 



K (/i - aj) 



{( k i)ie[N] '■ M-a/ = E^ ai } 

N 

\( k i)ie[N] '■ /u = E fe ^ + E Q 4 



i=i 

N 



i=l 



7.1 An example: A 2 
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Applying inclusion-exclusion, we get 



N N 

K(ji-ai) = {(fci) ie[JV ] : /i = J3fciO!i} - J3|{(*i)i 6 [jv] : /i = 5^*iOi, % = o} 



i=l 



N 



i=i 



JCJ 



Note that the function 



N 

■ (J. = fejaj, fej = if j € J | 



{(^)ie[iv]\j : X/ ^ ai } 

iG[JV]\J 



(78) 



is a vector partition function, corresponding to the matrix Mj with {ctj : j G J} as columns. We will 
denote this function by Kj(fi). With this notation, we can write 



7C[Af] JC/ 



(79) 



Denote by Cj the chamber complex associated to the partition function Kj. If Mj has full rank n, then the 
base cones whose common refinement is Cj are the positive hulls of the columns of the nonsingular n x n 
submatrices of Mj. As these submatrices are also nonsingular n x n submatrices of Ma„ = Mrjv], the 
base cones of Cj are also base cones of C n , and Cj is therefore a coarsening of the chamber complex for 
the Kostant partition function. If Mj does not have full rank n, then the base cones of Cj are the positive 
hulls of the columns of the maximal-rank submatrices of Mj. These cones are faces of the base cones for 
M J 4 n = Mnyi. The complex Cj will therefore be a coarsening of the restriction of C n to the positive hull 
of the columns of Mj (a lower dimensional complex). 

In view of all this, if \i stays strictly within any given cell of C n , it also stays in the same cell of Cj (for any 
J), and Kj(fx) is obtained by evaluating the same polynomial attached to that cell of Cj. Hence K q (fi) is 
given by polynomials over the relative interior of the cells of C n . Since Kj(/j,) has degree at most (™) in 
H (see remarks after Lemma l7~2t . equation d79l gives that K q {fx) is polynomial of degree at most (™) m A 4 
with coefficients of degree at most N = ( n ^ 1 ) in q. ■ 



7.1 An example: A 2 



The chamber complex C 2 has two top-dimensional cones: 



t~i = {«iai + a 2 a 2 : oi, a 2 > and a\ > a 2 } , 
t 2 = {a\ocx + a 2 a 2 : a±, a 2 > and a\ < a 2 } , 
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three 1 -dimensional cones: 



r 3 = {a(a\ + a 2 ) : a > 0} , 
T4 = {a\a\ : a\ > 0} , 
r 5 = {a 2 a 2 : a 2 > 0} , 



and the 0-dimensional cone 

r 6 = {0}. 

For /x = (hi, [12,^3) in the root lattice (in particular, /ii + /X2 + /X3 = 0), we get 

( (pi + - l)g 3 + 2g 2 
(/xi - l)g 3 + 2g 2 
(/xi - l)g 3 + + g 



q 
1 

I 



if n G n , 
if /x G r 2 , 
if // G r 3 , 
if fJL G T4 or /j, G T5 , 
if /x G r 6 , 
otherwise . 



(80) 
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